The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus for associating a plurality of objects in an image with each other and an image processing apparatus.
An object entering an area (a target monitoring area) is monitored by using an imaging apparatus such as a television camera (TV camera). Techniques of automatic monitor not by a human guard but by an apparatus or a system have been investigated.
For example, U.S. Pat. No. 6,445,409 (JP-A-2000-105835), the disclosure of which is hereby incorporated into this application by reference, describes that an input image captured with a camera is compared with a reference background image formed in advance, i.e., an image not showing an object to be detected, an area in the input image where a luminance value changes is detected by obtaining a difference between luminance values of each pixel, and a change in the position of the detected luminance value change area is judged for each processing frame to detect a suspicious person or an abnormal state in the target monitoring area. This process of detecting an area in the input image where a luminance value changes is called a subtraction method which is widely used conventionally.
With reference to FIGS. 8 and 9, description will be made on a process of the subtraction method.
FIG. 8 illustrates an example of how a luminance value change area is detected by the subtraction method. In this example, a luminance value change area in an input image sequentially input from an imaging apparatus is detected by the subtraction method.
An image 81 is an image sequentially input from an imaging apparatus, and an image 82 is an image prepared in advance and not showing an object to be detected (a reference background image). A subtractor 91 input with these two images 81 and 82 calculates a luminance value difference of each pixel to obtain a difference image 83.
Next, a binarizer 92 input with the difference image 83 executes a threshold value process for each pixel value of the difference image 83 by using a threshold value Th (e.g., an experimentally determined value, and Th=20). The binarizer obtains a binarized image 84 by setting a pixel value smaller than the threshold value Th to “0” and setting a pixel value not smaller than Th to “255”. In this manner, the subtractor 91 calculates an object 101 of a human shape taken in the input image 81 as an area 102 (where a luminance value of the input image changes) where a difference appears, and the binarizer 92 detects this area as an image 103 having a pixel value of “255”.
In this example, a pixel value of each pixel is calculated as eight bits, i.e., one pixel has a value from “0” to “255”
FIG. 9 illustrates an example of the procedure of a typical process of a monitoring method applying the subtraction method.
At an initializing Step, an external apparatus for executing the monitoring method by the subtraction method, variables, an image memory and the like are initialized (Step S21). At an image input Step, an input image of, e.g., 640 horizontal pixels and 480 vertical pixels is acquired from an imaging apparatus (Step S22). At a subtraction Step, a difference image (image 83) between luminance values of each pixel is calculated by comparing the input image (image 81) obtained at the image input Step with the reference background image (image 82) formed in advance (Step S23).
At a binarizing Step, a binarized image (image 84) is obtained by setting to “0” a pixel value (difference value) of the difference image (image 83) obtained at the subtraction Step and having a value smaller than a predetermined threshold value Th (e.g., Th=20) and by setting to “255” a pixel value not smaller than the threshold value Th (Step S24).
Next, at a labeling Step, clusters (image 103 as a change area) of pixels having the pixel value “255” in the binarized image (image 84) obtained at the binarizing Step are detected, and a number is assigned to each cluster to distinguish each cluster (Step S25). At an object existence judging Step, in accordance with the detection conditions such as a size, an area, a speed and a detected position, it is judged whether each labelled change area matches the detection conditions (Step S26). If the change area matches the detection conditions, it is judged that there is an object to be detected, and the flow advances to an alarm/detection information indicating Step (Step S27), whereas if the change area does not match the detection conditions, it is judged that there is no object to be detected, and the flow returns to the image input Step (Step S22).
At the alarm/detection information indicating Step, an existence of the object to be detected is notified to a guard by stimuli such as visual sense, auditory sense, smell sense, touch sense and taste sense, by using at least one means such as a monitor, an alarm lamp and an alarm buzzer (Step S27).
In the monitor system described above, a display device such as a monitor displays an image showing a detected object surrounded by a frame and a motion route, the image paying attention to an object detected under control of a zoom lens and a pan and tilt head of an imaging apparatus.
However, with the monitor system described above, a person (e.g., guard) searches another object associated with a suspicious object. This work is very complicated and inefficient.
More specifically, if a suspicious object is detected in a target monitor area and a guard views this object, the guard is required to view past images recorded in a predetermined image recording apparatus to find another object associated with the detected suspicious object, and to confirm whether the other object is also a suspicious object.
With reference to FIGS. 10 to 12, a specific example of this issue will be described.
FIG. 10 shows an example of an image 111 displayed on a display device of a monitor system.
In this example, if an object is detected in a target monitor area picked up by an imaging apparatus, for example, a circumscribed rectangle and a locus of the object are displayed. An entrance inhibition area 121 is set in the target monitor area, and if an object enters this area, it is judged that the object is a suspicious object. A thickness or the like of a frame of a circumscribed rectangle of the suspicious object is changed to display the object so as to distinguish the suspicious object from a simply detected object (i.e., an object outside the entrance inhibition area 121).
The image 111 shows the monitor results of objects, objects 122 and 123, in the target monitor area. The entrance inhibition area 121 is set at the upper right of the target monitor area.
FIG. 11 shows an example of an image 112 displayed on the display device after some time lapse from the state shown in FIG. 10.
Two objects 122 and 123 detected in FIG. 10 contact. In this case, the two objects are detected as one object 131.
FIG. 12 shows an example of an image 113 displayed on the display device after some time lapse from the state shown in FIG. 10.
An object 141 enters the entrance inhibition area 121. In this case, the object 141 and an object 142 are detected. Although the object 141 is judged suspicious because it enters the entrance inhibition area 121, the object 142 is not judged suspicious.
With this conventional method, for example if it is be judged from a position of an object whether the object is suspicious, this judgement is performed for each individual detected object. It is therefore necessary that a guard visually confirms another associated object (in the above example, the object passed a baggage) other than the object judged suspicious.